Fail Early, Fail Often: Towards Coherent Feedback Loops in Design Science Research Evaluation

We propose feedback loops that increase the coherence between evaluation activities in a design science research (DSR) process. While several scholars have proposed DSR cycles with frequent evaluation activities to provide timely feedback on design activities, the question of how to ensure coherence between these activities has remained largely unaddressed. Yet, coherence is essential to claim validity not only for the DSR artifact, but also for the DSR process. Based on a review of existing DSR literature, we propose an approach that ensures coherence between initial problem definition and final evaluation activities by explicating the notion of relevance underlying the DSR project, and between design and construction activities by creating situational design specifications. We exemplarily apply our approach to an ongoing DSR project. We conclude with a research agenda, where we build on the recent debate on generalizability in information systems to identify six fruitful avenues for further research.